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Status  Report:  Background  Subtraction  for 
Activity  Recognition 

I.  Introduction 

Activity  recognition  involves  analysis  and  monitoring  of  activities  performed  by  an  individual  or  by  a  group 
of  individuals.  The  initial  preprocessing  in  activity  recognition  system  involves  foreground  extraction.  We  have 
implemented  the  background  subtraction  method  proposed  by  Stauffer  and  Grimson  [1].  Further  a  technique  to 
compensate  for  the  camera  automatic  gain  control  (AGC)  effect  has  been  implemented.  The  problem  AGC  creates 
is  that  whenever  a  dark  or  bright  object  comes  in  the  field  of  view,  it  changes  the  illumination  of  complete  image 
which  results  in  failure  of  background  subtraction  techniques.  We  shall  illustrate  this  with  examples. 

II.  Stauffer-Grimson  Background  Subtraction  method 

Authors  of  [1]  modeled  the  pixel  process  by  the  mixture  of  adaptive  Gaussians.  This  method  accounts  for  the  fact 
that  a  pixel  can  see  multiple  surfaces  at  different  times.  The  pixel  process  is  defined  as  the  values  taken  by  the  pixel 
over  a  period  of  time.  A  new  pixel  is  represented  by  one  of  the  major  components  of  the  Gaussian  mixture  model, 
and  is  also  used  to  update  the  model.  Furthermore,  there  is  a  weight  associated  with  each  Gaussian  distribution. 
The  weight  of  the  matched  Gaussian  is  to  be  increased,  and  those  of  unmatched  are  to  be  reduced. 

W k,t  =  (1  —  Oi)cUk,t-l  #  (1) 

where  U0k,t  denotes  weight  for  k- th  Gaussian  at  time  t  and  M  is  equal  to  1  (or  0)  for  matched  (or  unmatched) 
Gaussian.  Also  mean  and  variance  cr2/^  for  matched  Gaussian  get  updated  based  on  the  observed  pixel.  The 
update  equations  are  as  follows: 

/it  =  (1  —  p)^t- 1  +  pXt  (2) 

C2t  =  (1  -  p)cr2t- 1  +  p(Xt  -  Pt)T(Xt  -  pt)  (3) 

where  Xt  denotes  pixel  intensity  at  time  t. 

Now  in  order  to  determine  which  Gaussians  of  the  mixture  are  most  likely  to  have  been  produced  by  the  backgroud 
at  every  pixel,  the  Gaussians  are  arranged  in  the  decreasing  order  of  u)k,t/&k,t-  The  background  Gaussians  are 
assumed  to  have  large  weights  and  less  variance.  The  first  few  Gaussians  are  chosen  as  the  background  Gaussians 
to  account  for  multi-modal  backgrounds.  After  segmentation  of  the  image  into  foreground  and  background  pixels, 
foreground  pixels  can  be  grouped  by  connected  component  algorithms. 
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III.  Stauffer-Grimson  Background  Subtraction  method  with  AGC  compensation 

The  camera  AGC  remains  ON  by  default.  Whenever  any  bright  or  dark  object  comes  in  the  view,  it  changes  the 
illumination  of  whole  image  causing  failure  of  background  estimation  methods.  It  was  shown  that  intensity  variation 
caused  by  camera  AGC  can  be  parameterized.  The  best  fit  curve  will  give  the  intensity  transfer  characteristics 
between  the  best  background  image  and  current  frame.  Fig.  la  shows  the  scatter  plot  of  pixels  in  the  best  background 
image  and  the  current  frame.  Fig.  lb  shows  the  corresponding  best  fit  curve  for  Red,  Green,  and  Blue  components. 
Also  new  update  equations  will  be: 

/A  =  (1  —  P)nt- 1  +  Pfb,t{^k,t- 1)  (4) 

where  (3  is  a  value  which  decides  the  adjustment  in  accordance  with  AGC. 

0-,2t  =  (1  -  /3)cr2t— i  +  (5) 

where  is  the  derived  transfer  function  relating  the  pixel  intensity  of  background  image  b  and  the  current  image 

t. 


(a)  (b) 

Fig.  1 :  Comparison  of  current  frame  with  background,  (a)  shows  the  scatter  plot,  and  (b)  shows  the  transfer  function 
relating  background  and  current  image  intensities. 


IV.  Experiments  and  Conclusion 

Fig.  2a  show  the  estimated  background.  Fig.  2b  is  the  430-th  frame  of  the  video.  The  figures  Fig.  3a  and 
Fig.  3b  show  the  foreground  extracted  by  Stauffer-Grimson  method  without  AGC  and  with  AGC  compensation 
respectively.  It  can  be  observed  that  AGC  compensation  has  improved  the  foreground  extraction.  It  shows  promising 
improvements.  However  sometimes  this  method  fails.  We  need  to  do  more  experimentations  to  gain  more  insights. 
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Fig.  2:  (a)Best  background  image,  and  (b)Current  image. 


Fig.  3:  Comparison  of  Stauffer-Grimson  method  with  and  without  AGC  compensation  for  foreground  extraction, 
(a)  Without  AGC  compensation,  and  (b)  With  AGC  compensation. 
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